Processes typically communicate through internode or intranode messages. There are many different types of standards that have been formed to attempt to simplify the communication of messages between processes, as message passing serves as an effective programming technique for exploiting coarse-grained concurrency on distributed computers. One such standard is the message passing interface (called “MPI”). MPI: A Message-Passing Interface Standard, Message Passing Interface Forum, May 5, 1994; and MPI-2: Extension to the Message-Passing Interface, Message Passing Interface Forum, Jul. 18, 1997. MPI is essentially a standard library of routines that may be called from programming languages, such as FORTRAN and C. MPI is portable and typically fast due to optimization of the platform on which it is run.
Message passing libraries can be used to provide parallel applications with communication service according to the MPI-2 standard specification. For internode communication the library uses network channels (e.g., Transmission Control Protocol/Internet Protocol (TCP/IP), Infiniband, Myrinet), and for intranode communication it uses a shared memory channel. Communication through the shared memory channel becomes an issue when the cluster node has more than one central processing unit (CPU) or CPU core such as in a multicore architecture.
In multiprocessor architectures that do not share system resources, messaging through shared memory from any process to another is equivalent, give or take the layout of the processors of a multiprocessor package or board. However, for multicore architectures this is not the case. Nevertheless, MPI libraries typically use a single memory copy routine to perform message passing.